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SPECIFICATION 



TITLE 



"METHOD AND DEVICE FOR THE REAL TIME CONTROL OF 
S PRINT IMAGES" 

BACKGROUND 

The inv e ntion preferred embodiment concerns a method and a device 
for real-time monitoring of print images. 
10 In the production of print works jobs , print errors can only be detected 

via purely visual observation at a later point in time due to the high speed with 
which print works lobs are moved in printing systems. The visual monitoring 
of print images is in particular difficult with continuous printing, since it is not 
possible to single out and to check an advance copy. If misprints are 
IS detected too late or not at all, high costs arise. 

However, incorrectly operating monitoring devices that trigger an error 
alarm can also incur unwanted costs due to the shutdown of the printing path. 

Therefore there exists a significant need for a robust method that 
reliably, safely and quickly detects print errors in the operation of a printing 
20 path. 

Video cameras with stroboscopic illumination are used for what is 
known as online print monitoring. The images supplied by these cameras can 
then be visually monitored and supplied to an automatic monitoring unit. 

A known method for automatic monitoring of print works is described in 

25 DE 199 40 879 A1. In this method, a reference image is generated or^ 
resp e ct i ve l y, provided if it already exists in digital form. A REAL {si^ image is 
detected by means of a stroboscopic light flash. The position of the REAL 
image is mapped on the reference image by means of a suitable correlation 
method. Since an exact superimposition of the reference image and the 

30 REAL image is not possible in practice, the reference image is sub-divided 
into sub-regions. The individual sub-regions can connect to one another 
without gaps or even an overlap. The differences of the color values of the 
pixels are determined in each sub-region. If the difference in a sub-region is 



-2- 

MARKED COPY 

greater than a predetermined tolerance threshold, the label "structure" is 
associated with the sub-region, and the label "color" is associated with the 
sub-region in the case that all differences in the sub-region are smaller than a 
predetermined tolerance threshold. The REAL image is compared with the 
5 desired color values in sub-regions with which the label co l or "color" is 
associated based on the REAL color values. In sub-regions with which the 
label structur e "structure" is associated, the average values or the sum of the 
amplitudes of all grey levels are determined and compared. 

This method has proved itself well in practice. However, there are 

10 fundamental disadvantages. Individual pixels of the REAL image are 
compared with the parameters of a sub-region that, given the label structure, 
do not precisely describe the color property. The quality of this monitoring 
method depends very much on whether the morphology of the printed image 
randomly coincides with the arrangement of the sub-regions. Since the 

15 individual regions are predetermined jn fixed, in particular long, narrow or 
short and wide sections of an image which possess a specific color property^ 
thev are not precisely monitored since they extend over a plurality of sub- 
regions, and in each sub-region the monitoring parameters to be determined 
have only a fractional influence. 

20 A method for comparison of print images is known from the document 

DE-A-199 40 879, in which currently-acquired images are compared with a 
reference image. The images to be compared are stored in pixel data. A 
system for monitoring of colors of print images m is known from the document 
US-A-6, 024,01 8, in which the image to be monitored is separated into regions 

25 that are then evaluated. 

SUMMARY 

Tho inv e nt i on i s thus bas e d on the It is an object to achieve a method 
and a device for monitoring of print images with which the reliability and 
quality of the monitoring is significantly increased relative to conventional 
30 methods o r, r e sp e ctive l y, devices. 
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Th e obj e ct i s ach io v o d via th e i nv e ntion doscr i bod in tho i nd e p e nd e nt 
c l aims. Advantag e ous e mbod i monts of tho inv e ntion ar e sp e c i fied in the sub - 
c l a i ms. 

In a method for monitoring of a print image, a real image is electro- 
5 optically detected and digitized in individual pixels. A reference image is 
provided that Is segmented into a plurality of segments such that respective 
Pixels in the respective segments exhibit approximately a same color property 
as the respective segments, a reference value describing said color property 
being associated with the pixels arranged in the respective segments. Color 
10 properties of the pixels of the real image are compared with the corresponding 
reference values of the reference image, and given a deviation from a 
predetermined threshold value, a corresponding pixel is marked as an error in 
a result image. 

BRIEF DESCRIPTION OF THE DRAWINGS 

15 Figure 1 shows schematically. In a flow diagram, a method for real-time 

monitoring of print images: 

Figure 2 illustrates schematically, in a flow diagram, a method for 
segmenting a reference image: 

Figure 3 shows a method for segmenting a reference image using 
20 some few pixels: 

Figure 4 illustrates a printing system in which the method of the 
preferred embodiment is used: 

Figure 5 illustrates a reference image: 

Figure 6 shows the segments of the reference image from Figure 5: 
25 Figure 7 illustrates a REAL image: 

Figure 8 illustrates a result image: 
Figure 9 shows a further reference image: 

Figure 10 illustrates the image from Figure 9 after the segmentation: 
Figure 11 shows the image from Figure 10 after the connection of 
30 individual segments: and 

Figure 12 illustrates the edges of the segments of the images from 
Figures 9 through 1 1 . 
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DESCRIPTION OF THE PREFERRED EMBODIMENT 

For the purposes of promoting an understanding of the principles of the 
invention, reference will now be made to the preferred embodiment illustrated 
in the drawings and specific language will be used to describe the same. It 
will nevertheless be understood that no limitation of the scope of the invention 
is thereby intended, such alterations and further modifications in the illustrated 
device, and/or method, and such further applications of the principles of the 
invention as illustrated therein being contemplated as would normally occur 
now or in the future to one skilled in the art to which the invention relates. 

The i nv e nt i v e method of the preferred embodiment for monitoring of 
print images comprises the following steps: 

electro-optical detection and digitization of a REAL image in 
individual pixels, 

use of a reference image that is segmented into a plurality of 
segments such that the segments respectively exhibit a specific 
color property, whereby a reference value describing the color 
property is associated with the pixels arranged in the respective 
segment, 

comparison of the color property of the pixels of the REAL 
image with the corresponding reference values of the reference 
image, whereby a corresponding pixel is marked as an error in a 
result image given a deviation above a predetermined threshold 
value, and whereby boundary regions of the segments are not 
considered in the comparison. 
In the i nv e ntion preferred embodiment , a reference image is used that 
is segmented into a plurality of segments such that the segments respectively 
exhibit a specific color property. Thus no randomly previously established 
sub-regions are used, but rather segments that in the reference image 
respectively comprise a region with an essentially identical color property. 
The segments thus reproduce the morphology of the image. Given this 
special design of the segments, significantly more precise reference values 
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can be used than is the case in conventional methods in which the sub- 
regions have been randonnly established. 

In the i nv e nt i v e method of the preferred embodiment , the pixels of the 
REAL image are thus compared with a very precise reference value, whereby 
5 deviations can be very reliably detected. 

Color properties in the sense of the following i nv e nt i on preferred 
embodiment can, for example, be grey values and/or color values. 

In particular a real-time monitoring of print images is possible with the 
i nv e nt i on preferred embodiment . 

10 According to a preferred method, boundary regions of the segments 

are not considered upon comparison of the pixels of the REAL image with the 
corresponding reference values of the reference image, whereby small 
register shifts (that are often unpreventable and are not viewed as errors by 
an observer) do not lead to unwanted error data. 

15 A result image in which the error data can be binarily {si€} associated 

with the individual pixels of the result image is generated with the inv e nt i v e 
image of the preferred embodiment . The result image can thus be shown as 
a binary image in which the regions are marked in which errors occur. Such a 
binary image can simply be shown on a display device and indicates the error 

20 locations of a printed image to an operator. The operator can h e r e by thus 
quickly and simply detect the errors and, in the event that it is necessary, take 
corresponding correction measures. 

Such a binary result image can also be very significantly compressed 
with known compression methods since it comprises only large-area binary 

25 (black/white) regions. This allows that the result images can be transferred to 
a monitoring station in real time over a data line with limited transfer capacity. 
The compressed result images can be uncompressed again at the monitoring 
station and shown on a display device. 

The invention preferred embodiment also provides a method for 

30 segmentation of a reference image in which regions with the same color 
property are determined, whereby these regions respectively form a segment. 
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A reference value that describes the color property of the respective segment 
tomography imaging system respectively associated with these segments. 

The inv e ntion i s subs e quently, ex e mplarily e xp l ain e d in dotai l us i ng 
draw i ngs. The draw i ngs show: 

5 F i gur e 1 sch e matioal l y, in a f l ow diagram a m e thod for r e a l- tim e 

mon i tor i ng of pr i nt imag e s, 

Figur e 2 schematioal l y, i n a f l ow diagram a method for s e gm e nting 

a r e f e r e nce i mag e , 

Figur e 3 a m e thod for s e gment i ng a r e f e r e nc e imag e us i ng som e 

10 f e w [s i c] p i x el s, 

Figur e A a printing syst e m in wh i ch th e i nv e nt i v e method i s us e d 

Figur e 5 a r e f e r e nc e imag e , 

Figur e 6 the sogmonts of th e r e f e r e nce i mage from Figure 5, 

F i gur e 7 a REAL imag e , 

15 F i gur e 8 a resu l t imag e , 

F i gur e 9 a further roforonco i mag e . 

F i gur e 10 th e imag e from F i gur e 9 aft e r th e s e gm e ntat i on. 

F i gur e 1 1 th e i mag e from F i gur e 10 aft e r th e conn e ct i on of 

i nd i v i dua l segm e nts, and 

20 F i gur e 12 th e e dg e s of th e s e gm e nts of th e i mages from F i gur e [s i c] 

0 through 1 1 . 

The invont i vo method of the preferred embodiment for real-time 
monitoring of print images is used in a printing system (Figure 4). Such a 
printing system comprises a printing device 1 . The i nv e nt i v e method of the 

25 preferred embodiment is typically used in high-capacity printers and in 
particular printers printing on continuous paper. Such a continuous paper is 
drawn from a paper roll 2 and \s supplied to the printing device 1 , A post- 
processing device 3 in which, for example, the continuous paper is cut into 
individual sheets is typically downstream from the printing device 1. The 

30 paper is conducted from the printing device 1 to the post-processing device 3 
along a paper track (schematically represented in Figure 3 by two roller pairs 
4). 
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A line camera 5 that is directed (with its objective) towards the printed 
paper web is arranged on the paper track. The passing paper can be electro- 
optically detected with such a line camera and these digital images can be 
individually created on the paper web ef for printed page s [sic] . These digital 
5 images respectively represent ae a REAL image. 

Instead of a line camera, a different electro-optical detection device can 
also be used such as, for example, a camera for acquisition of a two- 
dimensional image In combination with a stroboscope, whereby the paper 
web is illuminated with light flashes emitted by the stroboscope such that 
10 individual pages of the moving paper web are respectively recorded. 

The camera 5 is connected with an evaluation device 6 that is typically 
a computer with a storage device and a central calculation device. The 
evaluation device 6 is connected with a display device 7. 

The REAL image generated by the camera 5 is stored in an image 
15 storage in the evaluation device 6 (step S2), 

The position of the stored REAL image is determined relative to a 
desired position. This can occur using register markings or specific identifiers 
in the image itself. Diverse correlation methods for this are known in the prior 
art. An affine transformation with which the individual pixels of the REAL 
20 image can be mapped to the desired position is determined using this position 
determination (step S3). 

Afterwards the individual pixels of the desired image o r, rospoctivo l y, 
their color properties are compared with the reference values of a reference 
image (step S4) in a loop. In this comparison, the pixel that should be 
25 compared with the reference image is initially mapped to the corresponding 
location in the reference image by means of the affine transformations. The 
reference image is divided into segments. This division is explained in detail 
below. A reference image is associated with each segment. In this 
comparison, it is established in which segment the affine-transformed pixel 
30 lies, whereby the reference value associated with the segment is then used 
for the comparison. If the color property of the pixel of the REAL image 
deviates from the correspondingly-selected reference value by a 
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predetermined threshold value (result of the comparison: no), this means that 
the pixel does not possess the desired color property. In such a case, in a 
result image a pixel is at the corresponding position in the image Is associated 
with a value t»=»e that represents the error (step S5). If the color property of the 
5 pixel of the REAL image is within the range around the reference value that is 
predetermined by the threshold value (result of the comparison: yes), this 
means that this pixel possesses the desired color property, and the 
corresponding pixel in the result image is associated with a value that 
designates the correctness of this pixel. For example, in the result image the 

10 error values are set with a "1" and the correct values are set with a "0". 

Afterwards it is checked whether all pixels of the desired image have 
been compared with corresponding reference values (step S7). 

The result image is prepared in the steps step S8. Individual pixels or 
a few pixels that are contiguous and marked as wrong are set back to the 

15 correct value. A single pixel or a few contiguous pixels (whereby their number 
depends on the resolution of the image) are not detected by an observer of a 
printed image and are therefore nef not considered In the present method. 

The result image is shown on the display device 7 (step S9) so that the 
result image can be observed by the operator of the printing system. 

20 As an option it can be provided to compress the result image after its 

preparation in order, for example, to transfer it over a local network to a 
monitoring station at which the result image is decompressed and shown on a 
display device. It has been shown that the binary result image, which is 
typically comprised of large-area regions with error values o r, r e sp e ct i v e ly, 

25 correction values, can be very significantly compressed and therefore can be 
quickly and simply be transferred as a small amount of data, even over data 
lines of lower data capacity. 

The color properties can be represented by grey values and/or by color 
values in the method described above. If color values are used, a color 

30 property can thus be described by a plurality of values, if, for example, the 
color property is represented in RGB space, a color value for red, green and 
blue are fsi^ to be specified for each color property. Given such multi- 
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dimensional color properties, an interval value is used as a threshold value. 
This can, for example, be a specific Euclidian interval in the color space. 
However, it can also be appropriate to correspondingly vary the interval 
according to human perception, which is developed significantly differently 
5 with different colors. For example, for this the RGB data of the REAL image 
are transferred into a color space which takes into account the properties of 
human color interval perception (for example CIELa*b*) 

The desired values are then likewise provided in such a color space so 
that the Euclidian interval can also be used here. 
10 However, there are also color interval measures that cannot be 

calculated in a Euclidian manner. Here a more complex calculation is then 
necessary. The determination of these interval measures is established in 
standard lettering. However, the bases are nevertheless specially-selected 
color spaces. 

15 In [[a]] the preferred embodiment of the inv e ntion , the boundary regions 

of the segments are not considered in the comparison of the pixels of the 
REAL image with the corresponding reference values in step S4. This is 
appropriate since, in spite of the affine transformations, remaining congruence 
errors can arise. These can arise due to uncertainties of the spatial 

20 determination or non-linear variations of the REAL and desired images 
against one another, for example via hygroeexpansivity fsi^ or sagging. 
This means that individual pixels in the boundary region could be incorrectly 
associated with an adjacent segment, whereby a false evaluation of the pixel 
would result. These problems in the boundary region are thus remedied by 

25 the non-consideration of the boundary region. The width of the boundary 
region depends on the resolution of the reference image. Suitable widths of 
the boundary region lie in the range from 1 to 10 pixels, preferably in the 
range of 1 to 4 pixels. 

The association of the reference values is achieved in a program- 

30 technical manner in that a label is associated with each segment and the color 
property is associated with each label. If the color property is a grey level, this 
association can, for example, be represented according to the following table: 
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Label 


Grey level 


0 


nop 


1 


100 


2 


130 


3 


215 


4 


190 


5 


160 


6 


235 


7 


80 


8 


55 


9 


30 


10 


255 



The label 0 is associated with the boundary regions and a code "nop" 
(which means "no operation") is associated with the label 0 instead of a grey 
S level. If a pixel lies in the boundary region, upon comparison the code for "no 
operation" is hereby invoked, whereby the comparison is not executed. The 
corresponding grey levels are respectively invoked for the further labels 1 - 
10. In the comparison itself, the absolute value is established between the 
grey level of the reference value and the grey level of the pixel to be 
10 compared^ and it is checked whether this absolute value is smaller than the 
threshold value. If this is the case, the grey value of the pixel thus lies in the 
desired range and the correct value is set in the result image. Otherwise the 
error value is set in the result image. 

If color values are used instead of the grey values, a set of color values 
15 that describe the respective color are respectively associated with each label. 

A method for segmenting a reference image is subsequently explained 
(Figure 2). A reference image must initially be provided (step S10). The 
provision o r, resp e ct i v el y, generation of a reference image can occur in that 
an error-free printout of the image is recorded with the optical detection device 
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5 (which is also used to record the REAL image) in order to generate a digital 
image file from the image. 

On the other hand, in the event that the image to be printed already 
exists as a digital image file, it is also possible to use this image file directly. 
5 However, it is h e r e by appropriate to adapt the resolution (I.e. the number of 
the pixels per lengthwise unit In each row and column) of this image file to the 
resolution of the REAL image. As a rule, the resolution of the REAL Image 
might be somewhat less than that of the image file serving as a master copy, 
which is why the resolution Is reduced In a corresponding manner by means 
10 of suitable and known interpolation methods. 

Afterwards^ contiguous regions In the reference image are determined 
that possess approximately the same color properties, whereby one such 
region respectively forms a segment (step S11). This can, for example, be 
executed according to the following: 
15 - The pixels are individually, respectively associated with a 

segment, whereby the pixels in each row j (Fig. 3) are 
processed from left to right, and the individual rows in 
succession from top to bottom. 

The reference values of the three adjacent pixels in the row 
20 above this pixel and the reference value of the adjacent pixel to 

the left of the pixel to be associated are read out from a pixel to 
be associated with a segment. If the pixels are arranged in rows 
j and columns i, relative to the to-be-associated pixel with the 
coordinates (i, J)^ the reference values of the pixels with the 
25 coordinates (i-, j-1 ), (i, J-1 ), (i+1 , j-1 ) and (i-1 , j) are read out. 

It is subsequently determined which of the four reference values 
is most similar to the color property of the pixel to be associated. 
If the difference of this reference value and the color property of 
the pixel to be associated is less than . a predetermined 
30 threshold, the pixel to be associated is associated with the 

segment that contains the pixel whose reference value is 
nearest to the color property of the pixel to be associated. 
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This association occurs in that the pixel to be associated fsic} 
with the label of this segment is plotted in the reference image. 
If the color property of the pixel to be associated d i ff e r [s i c] differs from 
the closest reference value by more than the threshold, this pixel can be 
5 associated with none of the adjacent segments. This pixel forms the core for 
a new segment, whereby a new label of the association table is generated 
and this new label is registered in the reference image at the location of the 
pixel. 

The color property of the one pixel that has initiated the formation of 
10 the new segment is Initially associated with the new label in the association 
table. This color property can be associated with this label as a reference 
value (step S12). Alternatively it is possible to use the average value of the 
color properties of the individual pixels of a segment as a reference value. 
Upon addition of a new pixel to a segment, its color property is h e r e by 
15 averaged with the previously determined reference value of the segment with 
the corresponding weighting. 

If the reference image is completely segmented, the reference image is 
comprised of contiguous regions whose pixels are respectively associated 
with a specific label. The label for the boundary region, namely the label "0", 
20 Is now associated with the pixels of the boundary regions of the segments 
(step S13). 

According to [[a]] the preferred embodimen t of tho i nvent i on , it is 
examined whether segments exist that comprise less than a predetermined 
number of pixels and therewith are smaller than a predetermined size. If such 

25 segments are present, it is checked whether the color properties of adjacent 
segments do not differ from the color property of this small segment by a 
predetermined second threshold. If this is the case, both of these segments 
are joined into a single segment, whereby a new label is associated with this 
new segment. The weighted average value from the reference values of both 

30 original labels fsic} is associated with this new label as a reference value. 
With the joining of small segments with further segments, the division into 
very small segments is prevented insofar as it is possible since such small 
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segments are not advisable for the monitoring of the print image, in particular 
when a boundary region is provided that is not checked. 

Figure 5 shows a reference image that comprises two rectangles. The 
upper rectangle is completely black and the lower rectangle exhibits a color 
5 gradient of black/white In the direction from bottom to top. Figure 6 shows the 
boundaries of the segments of the reference Image shown in Figure 5. The 
black rectangle forms a single segment 9. The lower rectangle with the linear 
color gradient Is divided into a plurality of stripe-shaped segments 9 whose 
reference value describes the average color property of the respective stripe, 

10 i.e. the average brightness o r, r e sp e ctive l y, the grey values of this stripe. 
Figure 8 shows a REAL image in which certain regions 8 are not correctly 
printed. The In the result image (Figure 8) that has been determined 
according to the method explained above {sic} these incorrectly-printed 
regions 8 are shown in black and the remaining area of the result image is 

15 white. An operator of the printing system who sees the black regions of the 
result image Immediately recognizes that a misprint exists and can initiate 
suitable measures to correct the misprint. 

Figure 9 shows a further reference Image. Figure 10 shows the 
reference image from Figure 9 after the segmenting according to step S1 1 . A 

20 specific color property Is associated with each segment. The Individual 
segments are here respectively represented by the color property that, in the 
present case, is a grey level. However, the representation of the color 
properties here occurs with false colors, meaning that the brightness of the 
individual segments in Figure 10 allows no conclusions about the actual grey 

25 level of the respective segment. Many small "specks" that respectively form a 
segment are recognizable in Figure 10. 

Figure 1 1 shows the image segmented according to Figure 10 after the 
Joining of segments according to the step SI 4. Here it is cleariy recognizable 
that many regions with small differing specks have been connected into large- 

30 area, uniform regions. 

The image according to Figure 11 was processed further in that, 
according to the step SI 3, the label 0 has been associated with the boundary 
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regions that have been detected. The boundary regions are shown white in 
Figure 12. The remaining regions are shown black. Using Figure 12, one can 
easily recognize that the segmentation corresponds to the original 
morphology (Figure 9) of the image , [sioL As is explained above, a 
5 significantly better quality and reliability is h e r e by thus achieved in the 
automatic monitoring of printer works jobs. 

The i nv e ntiv e method of the preferred embodiment is executed on the 
printing system shown in Figure 4. The method can be realized as a 
computer program that is stored on the computer of the evaluation device 

10 such that it can be executed. This computer program can be stored on a data 
medium and can be executed on other printing systems. 

The i nv e ntion preferred embodiment can be summarized in brief 
according to the followjng[[:]L 

The quality in the automatic monitoring of print images in real time is 

15 improved w i th th e i nv e nt i on in that a reference image is used that is 
segmented such that the pixels of the segments possess approximately the 
same color property. The segments of the reference image hereby 
approximately reproduce the morphology of the reference image, whereby a 
reference value that describes the color property of the segment is very well is 

20 associated with each segment. The pixels of the REAL image are 
respectively compared with the reference value of the corresponding 
segment. This comparison is very reliable due to the high quality of the 
reference value. 

While a preferred embodiment has been illustrated and described in 
25 detail in the drawings and foregoing description, the same is to be considered 
as illustrative and not restrictive in character it being understood that only the 
preferred embodiment has been shown and described and that all changes 
and modifications that come within the spirit of the invention both now or in 
the future are desired to be protected. 

30 
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Cla i ms 

I CLAIM AS MY INVENTION: 
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ABSTRACT OF THE DISCLOSURE 

In a method for monitoring of a print image, a real image is electro- 
opticallv detected and digitized In individual pixels, 'A reference image is 
provided tliat is segmented into a plurality of segments such tliat respective 

5 pixels in the respective segments exhibit approximatelv a same color propertv 
as the respective segments, a reference value describing said color propertv 
being associated with the pixels arranged in the respective segments. Color 
properties of the pixels of the real image are compared with the corresponding 
reference values of the reference image, and given a deviation from a 

10 predetermined threshold value, a corresponding pixel is marked as an error in 
a result image. 
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